Digital Camera with High Dynamic Range Mode of Operation

ABSTRACT

A digital camera is provided with a high dynamic range (HDR) mode of operation wherein a nominal exposure is automatically bracketed by one or more under exposures and one or more over exposures. The images thus obtained can then be used by suitable software to process the images to produce a final HDR image.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to provisional patent application No.61/042,444 filed Apr. 4, 2008, the disclosure of which is incorporatedherein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to digital photography and, moreparticularly, to a digital camera with high dynamic range mode ofoperation.

2. Background Description

High dynamic range (HDR) photography is becoming more popular because itpermits the photographer to capture a greater range of tonal detail thanthe camera is capable of capturing in a single photograph. Sensors fordigital cameras have become higher resolution with each new generationof sensor. The increased number of pixels correspondingly means adecrease in the size of the individual pixels for a given sensor size.Unfortunately, the higher resolution of the sensors is often notaccompanied by an increase in the luminance range of the sensors. Forexample, in a landscape where the sky is bright but the foreground isonly moderately bright, the photographer must make a determination as towhether the photograph should be properly exposed for the sky, makingthe foreground quite dark, or properly exposed for the foreground,blowing out all the detail of the sky. One solution has been to use aneutral density gradient filter attached to the lens which, in effect,lowers the exposure of the sky when the foreground is properly exposed.However, this well known approach is limited, and that is the reason forthe current interest in HDR photography which, up until recently, hadbeen primarily used in computer generated imagery.

The technique requires the photographer to provide a stable platform forthe camera, i.e., a tripod, and take a series of exposures of the samescene, typically three or five exposures, at different exposure values(EVs). It is important that the camera not be moved between thesuccessive exposures since the computer software that will merge theseries of photographs requires that the images be aligned. Some of theavailable computer software provide a limited alignment function toaccount for any misregistration between images. The computer softwaremaps luminance values at each corresponding pixel of the several images.Examples of current computer software that provides HDR merging ofimages are Photomatix Pro and Photoshop CS3 and CS4.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a digital camerawith a high dynamic range mode of operation to facilitate generating theimages needed to produce HDR images and photographs.

There is provided digital camera having an automatic, continuous shutterfunction and an exposure bracketing function. The camera includes a userinterface, typically in the form of an LCD screen and a navigationbutton, which allows the user to select such things a white balance, ISOvalue, picture quality, and the like. This user interface also allowsthe user to select various modes of operation. According to theinvention, a high dynamic range (HDR) mode of operation is providedwhich the user can select. In a preferred embodiment, the user is ableto select between a plurality of HDR modes, depending on the number ofimages to be captured when operated in an HDR mode. Once the HDR mode ofoperation is selected, when the user depresses the shutter release halfway, causing the camera to perform automatic focus and exposuremetering, the exposure value measured by the camera is temporarilystored as the nominal exposure. In other words, this nominal exposurevalue (EV) is the exposure the camera would use when taking a singleimage in normal mode, single frame of operation. However, this nominalexposure is automatically bracketed in the selected HDR mode. That is,predetermined exposure values above and below the nominal EV aredetermined by the camera when in HDR mode. Now depending on the exposuremode selected by the user, e.g., automatic, aperture priority, orshutter priority, the bracketed exposure values may have varyingaperture and shutter settings (automatic mode), varying shutter speeds(aperture priority mode), or varying apertures (shutter priority mode).However, in the preferred embodiment of the invention, when HDR mode isselected, the user selected exposure mode is overridden to be aperturepriority while in HDR mode. This is to ensure that the depth of field(DoF) of each exposure remains constant. If the particular HDR modeselected requires three exposures, then the bracketing on either side ofthe nominal EV is some predetermined EV for exposures above and belowthe nominal EV. If the HDR mode selected requires five exposures, thenbracketing on either side of the nominal EV is two, typically evenlyspaced, exposure values on either side of the nominal EV. Note here thatwhen the camera is in aperture priority mode, the bracketing EVs areaccomplished by varying the shutter speeds. Now, when the user fullydepresses the shutter release, the camera takes in continuous successionthe number of programmed exposures for the HDR mode selected.

In addition to overriding a user selected exposure mode to be aperturepriority mode when an HDR mode is selected, in the case of auto focuslenses, the focus is fixed during the successive exposures. Asmentioned, when the shutter release is pressed half way, the cameraperforms automatic focus and exposure metering. It is important thatlens focus remain constant for each of the successive exposures. This isaccomplished in manual HDR exposures by manually switching the lens tomanual focus after first acquiring focus in the autofocus function. Thepurpose of the present invention is to avoid the many manual steps thathave been required in taking multiple exposures for HDR photography;therefore, according to the invention, when the shutter release is fullydepressed, the autofocus function of the camera lens is disabled duringthe continuous sequence of exposures.

In an alternative embodiment of the digital camera according to theinvention, the shutter priority mode may be selected. The selectedshutter speed would be sufficiently fast to allow for hand holding thecamera. This combined with, for example, vibration reduction (VR) ofcertain Nikon lenses or image stabilization (IS) of certain Canonlenses, or other vibration reduction and image stabilization mechanismsof other lens manufacturers, should produce reasonably well alignedimages since the exposures would be made sufficiently fast to maintainproper alignment. Any variation in DoF can be compensated for incomputer processing of the images. While not the preferred embodiment,this embodiment might be used to enable hand held operation.

In yet another alternative embodiment of the invention, instead ofbracketing EVs on either side of a nominal EV, a first exposure can bedetermined as an exposure where the highlights are just being blown out,i.e., the image is over exposed. Bracketing would then take the form ofmultiple exposures, each of which are successively reduced in EV. Thenumber of such exposures can be predetermined in the camera or the usercan be given the option to preset the number of exposures.

In its simplest form, the camera would have a single HDR mode selectableby the user. In this mode, the camera would select an EV having asufficiently high shutter speed to allow for hand holding exposures forthe lowest bracketed EV. The camera would then operate in aperturepriority mode, taking at least three exposures at different shutterspeeds in continuous shutter operation, yielding three exposures withsuccessively different EVs.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, aspects and advantages will be betterunderstood from the following detailed description of a preferredembodiment of the invention with reference to the drawings, in which:

FIG. 1 is a simplified diagrammatic rear view of a digital single lensreflex (DSLR) camera of the type in which the invention can beimplemented;

FIG. 2 is an illustration of menu selections that may be made toimplement the HDR mode in the DSLR camera;

FIG. 3 is a flow diagram of the firmware in the DSLR camera implementingthe HDR mode of operation according to a preferred embodiment of theinvention;

FIG. 4 is a flow diagram of the firmware in the DSLR camera implementingthe HDR mode of operation according to another preferred embodiment ofthe invention; and

FIG. 5 is a flow diagram of the firmware in the DSLR camera implementinga simplified HDR mode of operation according to yet another preferredembodiment of the invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

Referring now to the drawings, and more particularly to FIG. 1, there isshown a rear view (i.e., the photographer's side) of a typical DSLRcamera 1. The camera is configured with a pentaprism view finder 101having an eyepiece 102 which allows the photographer to view a scene tobe photographed through the lens (not shown) attached to the front ofthe camera. Once the photographer has framed the scene, the photographis taken by pressing the shutter release button 103. In most modern DSLRcameras, pressing half way down on the shutter release button causes thelens to be automatically focused, allowing the photographer to controlthe point of focus. The exposure can be set by the photographer to be,for example, automatic, aperture priority, shutter priority, or manual.Other programmed modes may also be available, depending on theparticular DSLR camera. When the picture is taken, an image of thepicture is provided in the liquid crystal display (LCD) 104 on the backof the camera. The displayed image is only displayed for a short periodof time to minimize battery drain, but if the photographer wants todisplay a recently taken picture, a button 105 adjacent to the LCD 104may be pressed to temporarily display the most recently taken picture.If the photographer wants to review of a series of pictures, a cursorbutton 106 is provided which, when pressed on the right side, advancesthrough a series of pictures and which, when pressed on the left side,reverses the sequence of displayed pictures.

The LCD 104 is also used to display various menus which allow thephotographer to setup up the camera for various conditions and desiredoutput. For example, the photographer can select various ISO(International Organization for Standardization) setting, white balance,and resolution of the picture, among other things, using the menusdisplayed on the LCD 104. To this end, there are provided a series ofadditional buttons 107, 108 and 109 adjacent the LCD 104. One of thesebuttons, say button 107, may be designated the “Menu” button. When thisbutton is pressed, a menu screen is displayed on the LCD. Thephotographer can navigate through the menu and display additional menusby operating the cursor button 106. More specifically, the photographercan move up and down through various menu entries by pressing the topand bottom of the cursor button 106 and can move left and right throughvarious menu entries by pressing the left and right sides of the cursorbutton. Inside the cursor button 106 is a concentric button 110 which,when pressed, makes a selection of a particular menu entry thephotographer has highlighted with the cursor button.

In addition to using the menus displayed in the LCD, the photographer istypically provided with shortcut buttons and thumb wheels that allow thephotographer to quickly make selections and changes without having toaccess the menus. So, for example, a thumb wheel 111 is provided for thephotographer to operate in combination with a selected button or switchto change a setting, such as ISO, white balance, exposure mode, or thelike. The typical DSLR camera is provided with a switch (not shown)which is often located on the front of the camera for selecting betweensingle frame (S) or continuous frame (C) exposure. In S exposure mode,pressing the shutter release button 103 causes a single exposure to bemade, whereas in C exposure mode, multiple exposures are made as long asthe shutter release button is depressed. Also, the typical DSLR cameraprovides the photographer with the option of bracketing exposures of ascene, and this is accomplished manually by pressing a button 112.

In the practice of this invention, the user interface for setting theHDR mode utilizes one or more of the controls described above. The firstof these, using the menus, is illustrated in FIG. 2. These menus aredivided into a tab section 201, a menu option section 202 and a currentsetting section 203. Navigation up and down the tab section 201 and themenu options sections 102 is by pressing the top and bottom of thecursor 106, and navigation between the several sections is by pressingthe right and left sides of the cursor button 106. A setting for a menuoption is selected by pressing the button 110.

The first menu 21, according to the present invention, shows one of themenu options to be HDR, or high dynamic range. However, thecorresponding current setting is blank, indicating that the HDR mode isnot selected. In one embodiment of the invention, by making a menuoption selection, the photographer causes the second menu 22 to bedisplayed. Here the photographer is presented with three options in theexample illustrated. These are “None”, “HDR3” and “HDR5”. The option“None” is self-explanatory. The option “HDR3” means that three exposureswill be made, one under exposed, one nominally exposed as metered, andone over exposed. Likewise, the option “HDR5” means that five exposureswill be made, two at different levels of underexposure, one nominallyexposed, and two at different levels of overexposure. These are onlyexamples and different numbers of exposures may be chosen in thepractice of the invention, but three and five exposures represent themost practical application of the invention in this embodiment. Inaddition, the amounts of underexposure and overexposure may beselectable by the photographer. For example, in shutter priority mode,the underexposure and overexposure may be selected to be a full f/stopor fraction thereof. Correspondingly, in aperture priority mode, thedifferent shutter speeds may be selected to accomplish underexposure andoverexposure.

Alternatively and in addition to the menu selection method, thephotographer may select HDR mode of operation by simultaneouslyselecting continuous (C) frame exposure and pressing the bracketingbutton 112. Then by using the thumb wheel 111, the photographer mayselect between HDR3 and HDR5 modes.

FIG. 3 is a flow diagram which illustrates the logic implemented in thefirmware of the camera in the practice of the invention according to thefirst preferred embodiment. The process begins in function block 301where the photographer enters the mode menu on the camera and, infunction block 302, selects one of HDR3 or HDR5 modes. See FIG. 2. Asjust explained, the photographer can also access and select one of theHDR3 or HDR5 modes using the short cut of operating switches and thethumb wheel. The two are entirely equivalent. Next, a determination ismade in decision block 303 as to whether the shutter is half depressed.Remember, this operation causes the camera to automatically focus on adesired point in the image. At the same time, the metered light is usedto determine the nominal exposure in function block 304. A determinationis made in decision block 305 as to whether the HDR3 mode is selected.If so, the bracketing function determines the amount of underexposureand the amount of overexposure in function block 306. A determination ismade in decision block 307 as to whether the shutter release button hasbeen fully depressed. If so, the shutter is automatically activated forthree successive exposures, one underexposed, one nominally exposed andone overexposed, as indicated by function block 308. During thisprocess, the autofocus function of the lens is disabled. Returning todecision block 305, if the HDR3 mode is not selected, then the HDR5 modeis selected. The bracketing function determines the amount of twodifferent underexposures and the amount of two different over exposuresin function block 309. A determination is made in decision block 310 asto whether the shutter release button has been fully depressed. If so,the shutter is automatically activated for five successive exposures,two different underexposures, one nominally exposure and two differentover exposures, as indicated by function block 311. Again, during thisprocess, the autofocus function of the lens is disabled.

As previously mentioned the under and over exposures may be accomplishedin aperture priority mode, shutter priority mode or automatic exposuremode. The first mode, aperture priority, would be selected to maintain adesired depth of field (DoF). The second mode, shutter priority, wouldbe selected to ensure that pictures are taken with the least amount ofshake in hand held operation. The third mode, automatic exposure mode,would allow the camera to determine the combinations of shutter speedsand aperture openings on the basis of computed exposure values (EVs).The second mode, shutter priority, provides a certain advantage in thepractice of the invention since, by using this mode, the need for atripod or other highly stable platform can be eliminated, but this canresult in slightly different depths of field for each exposure. In manycases this can be compensated in computer processing software of the HDRimages. Generally, the best approach to HDR photography is to operate inaperture priority mode to maintain a constant DoF in each exposure.

In yet another alternative embodiment of the invention, instead ofbracketing EVs on either side of a nominal EV, a first exposure can bedetermined as an exposure where the highlights are just being blown out,i.e., the image is over exposed. Bracketing would then take the form ofmultiple exposures, each of which are successively reduced in EV. Thenumber of such exposures can be predetermined in the camera or the usercan be given the option to preset the number of exposures. This isillustrated in FIG. 4 where, in the illustrated example, the user wouldpreset the number of exposures for HDR mode in advance in function block401. With the number of exposures preset by the user, the HDR processbegins in function block 402 when the user enters the mode menu on thecamera. The user then selects the HDR mode in function block 403. Adetermination is made in decision block 404 as to whether the shutterbutton has been depressed half way. This is the indication thatautomatic exposure and focus is to be made. In function block 405, thecamera determines an exposure value which is just slightly over exposed.A determination is made in decision block 406 as to whether the presetnumber of exposures is greater than three. If so, the exposure isbracketed by a negative sequence of predetermined EVs or fractionsthereof (i.e., increasing shutter speeds) in function block 407. Whenthe shutter button is fully depressed, as determined in decision block408, the shutter is automatically activated in function block 409 forthe preset number of successive exposures, each exposure being at alower EV by a predetermined stop or fraction thereof. During thisprocess, the autofocus function of the lens is disabled.

Returning to decision block 406, if the present number of exposures isgreater than 3, the exposure is bracketed by a negative sequence ofpredetermined stops or fractions thereof in function block 410. When theshutter button is fully depressed, as determined in decision block 411,the shutter is automatically activated in function block 412 for thepreset number of successive exposures, each exposure being at a lower EVby negative sequence of predetermined EVs or fractions of EVs (i.e., asequence of faster shutter speeds). Again, during this process, theautofocus function of the lens is disabled.

It will be appreciated by those skilled in the art that the examplesgiven in FIGS. 3 and 4 can be modified. For example, while both of theseembodiments change the EV by a negative sequence of predetermined stopsor fractions of stops for a number of exposures less than or equal tothree and by a negative sequence of predetermined EVs or fractions of anEV for a number of exposures greater than three, the EVs for eachexposure can be different for each of these two specific examples. Forexample, a half or other fractional EV could be implemented forexposures greater than three. Likewise, less than two EVs could beimplemented for less than or equal to three.

Additionally, the embodiments illustrated in FIGS. 3 and 4 could befurther modified so that, when entering the HDR mode, aperture prioritymode is automatically selected, overriding any prior selection by theuser. As is well understood by those skilled in the art, having aselected aperture and bracketing EVs by changing shutter speeds insuresa constant DoF for each exposure. This embodiment, however, may requirethat HDR mode be best operated when the camera is mounted on a tripod.

In its simplest form, the camera would have a single HDR mode selectableby the user. In this mode, the camera would select an EV having asufficiently high shutter speed to allow for hand holding exposures forthe lowest bracketed EV. The camera would then operate in aperturepriority mode, taking at least three exposures at different shutterspeeds in continuous automatic shutter operation, yielding threeexposures with successively different EVs. This embodiment isillustrated in FIG. 5. The process begins by entering the HDR mode infunction block 501. In function block 502, a determination is made ofthe lowest shutter speed for hand held operation. This is typically afunction of the focal length of the attached lens and may vary as aresult of operating the zoom feature of a zoom lens. Generally, theslowest shutter speed for hand held operation is recommended to be thereciprocal of the focal length of the attached lens. So, for example, ashutter speed of 1/60 sec. is recommended for lenses of focal lengths 50to 55 mm. This may vary if the lens is equipped for vibration reduction(VR) or image stabilization (IS) or similar technology which would allowa slower shutter speed. The camera is set to aperture priority mode infunction block 503 based on the lowest shutter speed determined infunction block 502. This will ensure that when the under exposed imageis taken, it will be at a shutter speed that was determined in functionblock 502. A determination is made in decision block 504 as to whetherthe shutter button is half depressed. If so, the exposure value for anominal exposure is determined. Then, in function block 505, the nominalexposure value is bracketed by predetermined EV settings. Adetermination is made in decision block 506 as to whether the shutterbutton is fully depressed. If so, the shutter is automatically activatedfor three successive exposures in aperture priority mode. Again, it willbe appreciated by those skilled in the art that the embodimentillustrated in FIG. 5 can be modified by implementing, for example, adifferent number of successive exposures at different EVs. For hand heldoperation, as a practical matter, three successive exposures appears tobe an optimum number, given the speeds at which the shutter can beoperated in continuous mode.

While the invention has been described in terms of preferredembodiments, those skilled in the art will recognize that the inventioncan be practiced with modification within the spirit and scope of theappended claims.

1. A digital camera comprising: a user interface which allows a user toselect a high dynamic range (HDR) mode of operation; means for computingbracketing of a nominal exposure by one or more under exposures and oneor more over exposures of a scene to be photographed; and a shutter andaperture mechanism which, upon a single shutter activation,automatically makes in rapid succession the one or more under exposures,the nominal exposure and the one or more over exposures of the scene. 2.The digital camera according to claim 1, wherein the user interfaceallows the user to select between multiple HDR modes.
 3. The digitalcamera according to claim 2, wherein the multiple HDR modes are an HDR3mode and an HDR5, the HDR3 mode bracketing a nominal exposure by oneunder exposure and one over exposure, and the HDR5 mode bracketing anominal exposure by two under exposures and two over exposures.
 4. Thedigital camera according to claim 1, wherein the user interface allowsthe user to preset a number of exposures and the means for computingbracketing first determines an over exposure value, bracketing of thenominal exposure value being computed based on the determined overexposure value.
 5. The digital camera according to claim 1, furthercomprising means which, upon selection of the HDR mode of operation,automatically sets the camera to aperture priority mode of operation,overriding any other mode of operation previously selected by the user.6. The digital camera according to claim 1, further comprising: meansfor determining a lowest shutter speed for hand held operation as afunction of lens focal length; and means for setting the camera toaperture priority for an aperture for an exposure value based on thedetermined lowest shutter speed.
 7. The digital camera according toclaim 6, wherein upon a single shutter activation, automatically makingin rapid succession one under exposure, a nominal exposure and one overexposure of a scene.
 8. The digital camera according to claim 1, whereinthe camera includes a lens and the camera and lens implement anautofocus function and an exposure metering function, the digital camerafurther comprising means, in response to the single shutter activation,for disabling the autofocus function during the successive exposures. 9.A method of operation by a digital camera, comprising the steps of:generating a user interface which allows a user to select a high dynamicrange (HDR) mode of operation; when an HDR mode of operation isselected, computing bracketing of a nominal exposure by one or moreunder exposures and one or more over exposures of a scene to bephotographed; and upon a single shutter activation by the user,automatically making in continuous succession the one or more underexposures, the nominal exposure and the one or more over exposures ofthe scene.
 10. The method of operation by a digital camera according toclaim 9, further comprising the step of setting the camera to aperturepriority mode of operation, overriding any other mode of operationpreviously selected by the user, when the HDR mode of operation isselected by the user.
 11. The method of operation by a digital cameraaccording to claim 9, further comprising the steps of: determining alowest shutter speed for hand held operation as a function of lens focallength; and setting the camera to aperture priority for an aperture foran exposure value based on the determined lowest shutter speed.
 12. Themethod of operation by a digital camera according to claim 9, furthercomprising the steps of: generating a further user interface when theuser selects the HDR mode of operation which allows the user to preset anumber of exposures; and determining an over exposure value, bracketingof the nominal exposure value being computed based on the determinedexposure value.
 13. The method of operation by a digital cameraaccording to claim 9, wherein the camera includes a lens, the camera andlens implementing an autofocus function and an exposure meteringfunction, further comprising the step of disabling the autofocusfunction during the successive exposures in response to the singleshutter activation.
 14. The method of operation by a digital cameraaccording to claim 9, wherein the user interface allows the user toselect between multiple HDR modes.
 15. The method of operation by adigital camera according to claim 14, wherein the multiple HDR modes arean HDR3 mode and an HDR5, the HDR3 mode bracketing a nominal exposure byone under exposure and one over exposure, and the HDR5 mode bracketing anominal exposure by two under exposures and two over exposures.